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PRKP/KIE 


Because  of  the  urgent  security  requirements  in  many  existing  general-purpose 
operating  systems,  the  large  investment  committed  to  such  systems,  and  the  large 
number  of  protection  errors  embedded  m them,  the  problem  of  finding  such  errors  is 
one  of  major  importance.  This  report  presents  an  approa<  h to  this  task,  based  on  the 
premise  that  the  effectiveness  of  error  searches  can  be  great!,  increased  by  techniques 
that  utilize  "patterns,"  i.e.,  formalized  descriptions  of  error  types.  It  give0  a conceptual 
overview  of  the  pattern-directed  evaluation  process  and  reports  the  authors’  initial 
experience  in  formulating  patterns  from  the  analysis  of  protection  errors  previously 
detected  in  various  systems,  as  well  as  in  applying  the  pattern  directed  technique.  This 
study  is  part  of  a larger  effort  to  provide  securable  operating  systems  in  DoD 
environments. 
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I. 


INTRODUCTION 


This  report  deals  with  the  problem  of  improving  the  security  of  existing  generalized 
resource-sharing  operating  systems  by  finding  errors  in  the  protection  mechanisms  of 
those  systems.  This  task  has  come  to  be  called  "protection  evaluation"  (PE).  The  PE 
problem  is  of  obvious  importance  in  view  of  the  investment  existing  systems  represent, 
their  expected  lifetime,  and  their  insecurity.  It  is  well  known  that  current 
general-purpose  Operating  systems,  which  are  large  and  complex,  usually  contain  a 
large  number  and  variety  of  errors  even  after  having  been  in  service  for  years.  That 
these  include  security  errors  is  indicated  by  the  fact  that  skillful  penetration  efforts 
directed  against  these  systems  invariably  succeed.  The  task  of  improving  such  systems 
is  urgent,  since  many  of  them  are  installed  in  environments--governmental,  commercial, 
and  military  in  which  the  requirement  for  security  (in  terms  of  the  magnitude  of  losses 
from  accidental  or  intentional  violations)  is  strong  and  immediate.  These  losses  will  be 
reduced  in  proportion  to  the  cost-effectiveness  of  the  available  error-finding  tools. 

In  economic  terms,  debugging  is  and  always  has  been  one  of  the  most  important 
problems  in  the  computer  field,  and  considerable  effort  has  been  spent  in  rttempts  to 
reduce  it.  The  results  have  been  less  than  spectacular;  there  remains  a wide  gap 
between  the  effectiveness  of  the  most  sophisticated  currently  available  debugging  tools 
and  the  eventual  ability  to  prove  the  correctness  of  properly  structured  operating 
systems.  The  approach  presented  here  is  directed  at  narrowing  that  gap.  The  goal  is 
to  provide  a basis  for  the  development  of  tools  (for  at  least  the  PE  application  area) 
that  are  significantly  more  effective  than  currently  existing  tools  but  that  can  also  be 
available  in  the  relatively  short-term  future.  The  proposed  approach  is  more 
formal  than  traditional  debugging  methods  but  less  formal  than  logical/mathematical 
verification.  It  also  restricts  itself  to  static  representations  of  operating 
systems— listings,  accompanying  documentation,  and  information  derived  from  them 
(including  the  output  of  compilers  and  loaders) — and  is  intended  to  complement  dynamic 
methods  such  as  testing  and  monitoring  in  the  common  situation  where  on-line  access  to 
the  target  system  by  evaluators  is  not  readily  available.  (It  should  be  noted  that  in 
some  cases  flaws  detected  by  static  methods  must  be  further  analyzed  with  respect  to 
dynamic  operations  to  determine  whether  they  represent  actual  errors.) 
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Typically,  PE  is  carried  out  manually,  using  only  simple  debugging  tools  and 
rudimentary  aids  such  as  symbol  concordances.  Usually  PE  projects,  such  as  those  of 
Project  RISOS  at  Lawrence  Livermore  Laboratory*  or  at  System  Development 
Corporation  [Wei73],  are  organized  around  teams  ot  individuals,  ar.d  their  success 
depends  heavily  on  the  motivation  o<  these  individuals  as  well  as  on  their  skill  in  finding 
protection  errors,  in  other  words,  they  require  individuals  who  would  themselves  make 
good  penetrators  of  the  target  system.  This  means  they  must  have  not  only  an 
intimate  knowledge  of  that  System  but  also  a good  understanding  of  or  feel  for 
protection  error  possibilities  Efforts  to  systematize  this  work  have  dealt  primarily  with 
the  organization  of  the  project  staff  itself 


The  goal  of  the  approve'  presented  here,  in  contrast,  is  to  make  PE  both  more 
effective  and  more  economical  by  decomposing  t into  more  manageable  and  more 
methodical  subtasks,  n such  a way  as  to  drastically  reduce  the  requirement  for 
protection  expertise.  The  approach  is  caPed  "pattern-directed"  because  it  is  based  on 
an  analysis  of  how  forma:  zed  error  patterns  may  be  used  to  direct  and  systematize  the 
PE  task. 

Section  2 states  the  bazc  observations  from  which  the  approach  it  derived  and  the 
requirements  that  shape  the  techniques  and  tools  to  be  developed.  Section  3 describes 
the  formulation  and  dassit  cation  of  patterns  together  with  some  initial  experience. 
Section  d examines  the  application  of  pattern-directed  techniques  in  light  of  the 
requirements  stated  ea'  e',  and  draws  conclusion  about  the  form  such  techniques  must 
take.  Section  5 gives  an  examp  e of  the  apolication  ot  a pattern-directed  technique  to 
a particular  tyoe  of  error  and  relates  the  results  of  a preliminary  test  of  the  feasibility 
of  the  pattern-directed  approach 


•Private  commonicanon  wdh  Robert  P.  Abbott. 
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2.  BASIC  CONSIDERATIONS 


The  pattern-directed  approach  to  PE  is  based  on  two  observations: 

1.  Protection  errors  of  the  same  or  similar  types  appear  not  only  in  different 
functional  areas  of  the  same  model  of  an  operating  system,  but  also  in  different 
systems.  Furthermore,  there  is  reason  to  believe  that  the  number  of  types  of  basic 
vulnerabilities  is  fairly  small.  Some  authors  have  speculated  that  the  number  is  less 
than  ten  [And72,  McP74],  although  the  types  they  list  do  not  entirely  correspond. 
While  the  definition  of  error  type"  is  open  to  question  (a  reasonable  interpretation  is 
suggested  in  Section  3),  we  have  already  identified  additional  types.  We  believe, 
however,  that  the  number  of  basic  types  is  less  than  25. 

2.  The  effectiveness  of  a search  depends,  in  part,  on  the  degree  to  which  the 
object  or  type  of  objects  )eing  searched  for  are  well-described  or  well-defined.  In  the 
case  of  protection  errors,  we  have  experienced  and  witnessed  in  others  the  large 
difference  in  effectiveness  between  a "blind  search"  and  an  examination  directed  toward 
errors  of  a particular  type  described  by  a concise  pattern.  A typical  example  is  that  of 
the  protection  error  continually  overlooked--even  though  textually  adjacent  to  an  error 
found  weeks  or  months  earlier-  until  noticed  as  an  instance  of  a given  pattern*  We 
have  found  that  even  persons  with  no  previous  experience  in  protection  evaluation  can 
find  errors  when  given  a specific  pattern  to  guide  their  search. 

An  approach  suggested  by  these  observations  is  to  (1)  identify  the  basic  error 
types  and  formulate  the  patterns  representing  them,  and  (2)  develop  search  techniques 
capitalizing  on  these  patterns.  These  basic  activities  are  described  in  the  next  two 
sections. 

Two  important  requirements  must  guice  the  development  of  pattern-directed 
techniques: 


♦In  a case  with  which  one  of  us  (Bisbey)  is  familiar,  an  error  was  discovered 
just  three  instructions  away  from  one  which  had  been  previously  corrected. 
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1 They  must  be  widely  applicable,  which  implies  that  to  a large  extent  they  must 
be  general -purpose  with  respect  to  operating  systems.  Little  is  gamed  over  current 
methods  if  completely  separate  techniques  are  necessary  to  evaluate  each  new  system 
of  different  manufacturer  or  version. 

2 If  these  techniques  are  to  be  significantly  more  effective,  economica',  and 
reliable  than  existing  techniques,  evaluators  who  use  them  must  not  be  required  to 
possess  particular  expertise  in  protection,  nor  lo  develop  any  deep  understanding  of 
protection  errors,  nor  to  be  .ble  to  recognize  them  as  such  in  the  dispersed  or 
camouflaged  form  in  which  they  frequently  exist  In  the  sect  ons  that  follow,  the  word 
"evaluator"  will  be  assumed  to  denote  such  a nonexpert.  The  use  of  these  techniques 
must  not  require  evaluators  to  perform  pattern  recognition  activities  nearly  as  difficult 
as  those  currently  required  to  find  protection  errors.  An  evaluator  will,  of  course,  be 
assumed  to  be  familiar  with  the  internals  of  the  system  being  evaluated. 


The  effects  of  these  requirements  are  discussed  in  Section  A. 
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3.  PATTERN  DEVELOPMENT 


There  are  two  alternative  strategies  for  deriving  error  patterns:  either  deduce 

them  from  theoretical  considerations  or  infer  them  from  an  analysis  of  errors  that  have 
already  been  detected  during  PE  of  existing  systems.  The  latter,  empirical  approach 
has  been  adopted  because  it  appears  to  offer  a greater  assurance  of  success  in  less 
time,  because  a substantial  number  of  such  errors  already  awaits  collection  and  analysis, 
and  because  we  believe  a methodical  collection  and  analysis  of  such  errors  is  a valuable 
undertaking  in  its  own  right  (e.g.,  to  develop  a manual  of  "good  design  practices  ). 

The  material  from  which  patterns  are  derived  are  "raw  errors,"  descriptions  of 
security  errors  found  in  various  operating  systems,  usually  expressed  very  informally 
and  in  terms  specific  to  the  particular  systems  in  which  they  were  found.  Our  collection 
currently  contains  raw  errors  from  the  0S/360,  GCOS,  Multics,  TENEX,  and  Exec-8 
systems.  The  following  is  an  example  of  a raw  error,  exactly  as  collected: 

"Snap  Dump  is  a supervisor  routine  for  providng  printed  core  dumps  of 
memory.  The  routine  consists  of  nine  nonresident  modules,  each  of  which  is 
separately  fetched  and  executed,  and  one  resident  module  wh'ch  remains  ir 
main  storage  for  the  entire  dump  process.  The  resident  module  (IEAQAD0A)  is 
loaded  by  the  first  segment  of  Snap  Dump  and  contains  several  format  and 
output  subroutines  used  by  the  other  modules.  The  error  is  that  if  a user 
names  his  program  IEAQAD0A,  his  program  will  be  given  control  in  privileged 
mode,  instead  of  the  system  program  of  the  same  name." 

A more  precise  representation  is  needed  than  the  unconstrained  narrative  in  which 
errors  are  first  obtained.  The  formulation  of  patterns  should  facilitate  both  their 
classification  and  their  application.  This  implies  that  patterns  should  be  complete  and 
concise  representations  of  errors,  cast  in  a standardized  form  and  notation. 
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Wl,h  ,hM's  ,n  m,nd'  wo  re2ard  a patten  f.rot  of  all  as  a vet  of  independent 
^condition.,,  pre  j ate  that  express  properties  of  or  relations  among  drt  net  objects  or 
features"  that  can  be  identify  or  recognized  m the  s,stem.  The  condition  set  of  a 
pattern  i minrra  n ti  e ef’re  that  f any  were  remo  ed  the  pattern  would  no  longer 
represent  a potential  f--or  h.e.,  an  error  can  be  corrected  by  changing  any  one  of  the 
conditions  that  imply  t).  Ine  following  are  examples  of  condition*,: 

The  calling  procedure  has  write- access  to  cel!  X" 


"The  value  of  oarameter  Y is  critical  to  procedure  P. 


t"e  address  of  W is  calculated  as  a function  of  7." 

"Procedure  A calls  procedure  Q.“ 

Control  is  passed  to  B in  the  environment  of  A." 

initially,  to  maintain  a clear  connection  between  a pattern  an  j the  error  from  which 
it  was  derived  and  to  avoid  overlooking  poss  ble  areas  of  application  of  that  pattern,  it 
is  important  to  express  it  in  terms  specific  to  its  source  operating  system.  For  this 
reason  the  initial  pattern  is  called  a "raw  pattern."  The  following  is  a raw  pattern  <or 
the  above  error: 

1.  Load  is  called  by  Snap  Dump  to  return  the  core  address  of 
IEAQAD0A 

2.  It  is  critical  to  Snap  Dump  that  the  module  loaded  is  the  actual 
system  module  IEAQAD0A, 

3.  The  identity  of  the  module  loaded  is  not  verified  by  either  Load  or 
Snap  Dump. 

More  formal  and  concise  pattern  notation  and  terminology  are  being  developed; 
these  will  be  reported  in  a subsequent  document. 


Gi>en  a raw  error,  it  is  often  difficult  to  write  down  a pattern  that  satisfactorily 
captures  the  essence  of  the  error.  First,  of  course,  the  error  description  must  be 
thoroughly  comprehended,  eg.,  in  terms  of  how  the  error  could  be  exploited  by  a 
Know  edgeabie  penetrator.  Tms  requires  familiarity  with  the  operating  system  context 
in  which  it  occurred.  Even  then  it  may  not  be  clear  precisely  what  policy  is  being 
violated  and  thus  what  conditions  should  constitute  the  pattern.  Consider  the 
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"oass-through"  problem,  for  example  [McP74],  A supervisor  procedure  P may  be 
orogrammed  to  omit  the  validity  check  for  a critical  input  parameter  X when  called  by 
other  supervisor  procedures,  assuming  that  X is  a p,  operly  maintained  system  data 
element  in  such  cases  Under  the  assumption  that  P checks  X,  another  supervisor 
procedure  Q calls  it  with  an  argument  for  X that  has  been  user-specified.  The  policies 
associated  with  P and  Q are  inconsistent.  In  such  cases,  in  which  different  but  equally 
val'd  policies  can  be  postulated,  the  same  raw  error  leads  to  more  than  one  pattern. 
Conversely,  of  course,  many  raw  errors  can  result  in  similar  initial  patterns. 

As  an  error  search  criterion,  a raw  pattern  is  directly  applicable  only  to  operating 
systems  that  share  the  policy  violated  by  that  error  and  in  which  the  features  of  that 
pattern  are  known  by  the  same  names.  Even  then,  it  may  apply  only  to  a particular 
functional  area  such  as  input/output  control,  and  miss  similar  errors  in  another  area 
such  as  interprocess  communication.  To  broaden  the  applicability  of  a pattern,  its 
expression  must  be  generalized  by  substituting  more  generic  names  or  more  abstract 
features  for  more  specific  ones  or  by  deleting  qualifying  details  without  affecting 
essence  of  the  conditions  themselves.  The  same  concept,  such  as  the  call  on  a 
privileged  system  procedure  by  an  unprivileged  user  procedure,  may  be  known  by 
different  names  (such  as  "MME,"  "JSYS,"  and  "SVC")  m different  systems.  Classes  of 
similar  objects,  such  as  bytes  or  blocks  of  physical  storage,  pages,  segments,  simple 
variables,  structured  variables,  and  files  (to  give  an  extreme  example),  can  be  regarded 
as  instances  of  a more  abstract  object,  in  this  case  the  "abstract  cell,"  something  that 
has  a name  and  holds  information  (its  value).  The  benefit  of  generalizing  is  that  the 

generalized  pattern  aDplies  to  a correspondingly  wider  class  of  errors  in  a wider  class 
of  systems. 

The  following  is  a generalization  of  the  raw  pattern  discussed  previously: 

1.  Supervisor  procedure  A is  called  by  supervisor  procedure  B to 
return  the  core  address  of  a procedure  or  data  element  C having 
name  N. 

2.  It  is  critical  to  B that  C is  the  bona  fide  system  element  named  N. 

3.  The  identity  of  C is  not  verified  by  either  A or  B. 

Here  the  names  of  the  specific  routines  have  been  replaced  parametr.cally. 

Conversely,  the  more  general  the  pattern  and  the  broader  its  applicability,  the  less 
directly  relevant  it  will  be  to  particular  functional  areas  of  particular  systems  and  the 
less  immediate  utility  it  will  have  as  a search  criterion,  since  its  features  must  first  be 
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identified  with  as  many  as  possible  of  those  of  the  target  system.  This  is  discussed  in 
the  next  section.  The  opposite  of  generalizing  a pattern  is  "instantiating"  it  by 
substituting  examples  or  instances  for  one  or  more  of  its  features.  Just  as  the  same 
pattern  can  have  many  generalizations,  a given  (non-raw)  pattern  potentially  has  many 
instances. 

The  derivation  of  raw  patterns,  their  generalization,  and  the  instantiation  of 
generalized  patterns  toward  other  systems  and  functional  areas  all  add  new  elements  to 
the  lattice  of  patterns  formed  by  the  relation  "generalization  of"  and  its  converse, 
instance  of,  with  the  more  abstract  patterns  at  the  top  and  the  more  concrete  ones  at 
the  bottom.  As  this  structure  grows,  major  substructures  may  emerge,  at  least  below 
some  level  of  abstractness.  If,  as  is  also  expected,  the  search  techniques  determined  to 
be  appropriate  for  the  patterns  of  each  such  substructure  are  also  similar,  then  a 
reasonable  basis  will  have  been  provided  to  define  distinct  major  "error  types.” 
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4.  DEVELOPMENT  AND  APPLICATION 
OF  PATTERN- DIRECT  ED  TECHNIQUES 


Detecting  errors  in  a set  of  target  information  implies  some  Kind  of  comparison 
process  between  the  target  and  the  correctness  or  error  criteria.  The  comparison 
need  not  be  direct;  various  transformations  may  be  applied,  as  practical,  to  either  the 
criteria  and  the  target  to  bring  them  into  a suitable  form,  as  long  as  essential  properties 
are  preserved  In  the  case  of  pattern-directed  PE,  the  target  is  a set  of  operating 
system  source  programs  and  specifications;  the  criteria  are  the  error  patterns;  and  the 
comparison  process  is  essentially  one  of  “pattern  recognition,"  in  the  sense  of  an  ability 
to  detect  instances  of  errors  embedded  or  camouflaged  in  a system. 

Conceptually,  the  ideal  tool  is  a general-purpose  "protection  evaluator,"  a computer 
program  that  not  only  could  be  applied  to  a wide  class  of  operating  systems  but  could 
also  reliably  detect  a wide  class  of  errors.  The  inputs  to  such  a program  would  be 
representations  of  the  patterns  for  the  error  types  covered,  together  with  a 
representation  of  the  target  operat  ng  system.  The  program  would  compare  the  target 
representation  with  the  given  patterns  by  searching  it  for  ah  combinations  of  features 
related  in  one  of  the  ways  specified  in  some  pattern,  and  would  report  every  such 
combination  found.  With  this  concept,  PE  is  regarded  as  consisting  of  two  subtasks: 

1.  Normalizing  the  target  system  by  extracting  the  information 
relevant  to  the  evaluation  and  representing  it  in  the  form  required 
by  the  comparison  program. 

2.  Executing  the  comparison  program. 

Such  an  ideal  is  clearly  out  of  reach.  There  exists  no  model  into  which  the 
protec  .ion-relevant  features  of  existing  systems  can  be  mapped  and  in  which  they  can 
be  related  for  comparison  with  given  patterns,  general  enough  to  apply  to  wide  classes 
of  errors  and  systems.  It  is  even  difficult  to  determine  with  precision  which  elements 
of  existing  systems  are  relevant  to  protection  and  which  are  not.  Much  research  is  now 
being  done  on  the  question  of  what  actually  should  constitute  a protection  "kernel" 
[Pan74],  including  the  effort  to  identify  a kernel  for  Multics*  and  efforts  to  design  new 
systems  based  on  this  notion,  such  as  Hydra  [Wul74]  and  the  UCLA-VM  system  [Pop74T 


♦Private  communication  with  Jerome  Saltzer. 
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Nevertheless,  the  goal  of  developing  pattern-directed  techniques  and  tools  to 
systematize  and  automate  PE  -oi.ia'ns  valid.  We  must  investigate  what  the  requirements 
for  these  techniques  stated  in  Section  2 imply  about  their  form,  application,  and 
development. 

First,  the  requirement  for  general-purposeness  with  respect  to  operating  systems 
carries  an  obvious  implication:  there  must  exist  some  generalized  set  of  terminology--a 
comparison  language"--in  which  the  techniques  are  specified  and  in  which  the  error 
patterns  are  expressed.  To  apply  these  techniques  to  a given  system,  it  is  then 
necessary  that  a correspondence  be  established  between  the  objects  and  terminology 
of  he  comparison  language,  i.e.,  between  the  features  of  the  given  patterns  and  their 
instantiations  in  the  target  system.  Either  the  features  of  the  patterns  must  be 
instantiated  to  the  concepts,  objects,  and  terminology  of  the  target  system  or  the  target 
system  must  be  represented  in  terms  of  the  comparison  language,  or  an  intermediate 
comparison  framework  must  be  established  and  transformations  performed  in  both 
directions.  If  no  error  possibilities  are  to  be  overlooked,  then  all  the  instances  of  a 
given  pattern  feature  in  the  target  system  must  be  identified. 

If  one  uses  the  term  "features"  to  refer  to  objects  that  have  concrete  and  typically 
localized  representations  in  the  target  system  description  (e.g.,  variables,  procedure 
calls,  critical  parameters),  then  identifying  the  relevant  features  in  the  target  system  is 
only  part  of  the  problem.  The  other  part  is  to  determine  whether  any  of  the  relations 
among  these  features  are  those  indicated  by  the  conditions  of  an  error  pattern.  The 
second  requirement,  i.e.,  that  evaluators  need  not  have  a talent  for  recognizing 
protection  errors  and  that  difficult  pattern-recognition  processes  must  not  be  involved, 
makes  it  essential  that  the  search  for  an  error  be  decomposed.  The  search  through  the 
target  system  code  (or  some  representation  of  it)  for  a single  dispersed  collection  of 
instances  of  feaiures  in  some  given  relation  must  be  replaced.  Instead  we  must  require 
only  independent  searches  for  individual  instances  of  features  in  the  target  system. 
This  implies,  of  course,  that  the  Output  of  these  searches  must  include  simple 
specifications  of  the  contexts  in  which  the  feature  instances  were  found.  The  needed 
feature  context  is  determined  from  the  relations  expressed  in  the  patterns  and  is  used 
to  determine  whether  the  features  found  actually  satisfy  these  relations.  Such 
searches  can  often  be  mechanized,  as  seen  in  the  example  given  in  the  next  section. 

The  search  output  'Onstitutes  the  input  to  a separate,  methodical  comparison 
process  in  which  the  properties  of  the  feature  instances  found  are  examined  to 
determine  whether  actual  (potential)  error  conditions  exist.  Obviously,  the  comparison 
is  still  not  a direct  one,  since  a translation  must  be  made  between  the  generalized 
relations  expressed  in  the  patterns  and  the  descriptions  of  feature  instances  provided 
as  input.  Again,  in  general  the  choice  must  be  made  between  expressing  the  search 
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results  in  the  comparison  language  and  instantiating  the  reference  properties.  The 
former  is  required  for  a system-independent  comparison  algorithm. 

The  above  considerations  have  led  us  to  an  evaluation  process  consisting  of  two 
steps  that  are  similar  but  more  straightforward  than  the  two  required  for  the  ideal 
evaluator  described  above. 

1.  "Feature  extraction,"  involving  the  instantiation  of  generalized 
features,  the  search  for  instances  of  these  features  in  the  target 
system,  and  the  description  of  their  relevant  contexts. 

2.  Comparison  of  combinations  of  feature  instances  and  their  contexts 
with  the  features  and  relat.ons  expressed  in  the  given  patterns 

The  nature  of  the  technique-  and  toois  to  be  developed  has  become  more  apparent 
They  consist,  for  a given  set  of  error  patte  ns,  of  a ■ et  of  generalized  directives  for 
searching  an  operating  system  for  instances  of  the  features  of  these  patterns  and 
describing  the  instances  found,  as  wet!  as  formal  or  informal  procedures  for  evaluating 
the  results  of  the  search  with  respect  to  the  given  properties  and  relations. 

In  view  of  the  inherent  problem.s,  an  effort  to  de  velop  such  tools  would  still  appear 
to  be  too  ambitious  were  it  not  for  the  simple  Ofcser  afion  that  it  is  not  nece-  -.ary  to 
have  an  integrated  pr>  >cage  that  (1)  contains  directives  for  a large  number  of  error 
patterns,  (2)  includes  a single  general-purpose  comparison  algorithm,  and  (3)  is  bared 
on  a single  comparison  language.  Instead,  a set  of  relatively  simple  packages  can  be 
de  veloped,  each  tailored  to  a particular  error  type  of  interest.  This  means  that  instead 
Of  requiring  general  solutions  to  the  problems  discussed  abo  e,  the  approach  requires 
only  solutions  to  problems  local  to  each  error  type.  The  directives  for  instantiating  and 
identifying  the  features  of  some  patterns,  and  for  describing  their  contexts,  are 
relatively  easy  to  formulate,  and  them  comparison  procedures  relatively  easy  to  specify. 
The  directives  and  procedures  of  each  search  package  can  be  designed  to  best  suit  that 
package  alone.  A set  of  techniques  enabling  protection  evaluators  to  search  an 
Operating  system  economically  and  reliably  for  instances  of  even  an  incomplete  set  of 
types  of  potential  errors  would  be  extremely  valuable.  A reasonable  approach  is  to 
start  with  those  error  types  for  which  the  payoff,  in  terms  of  the  likely  cost  of  such 
errors  in  existing  operating  systems,  is  greatest  relative  to  the  effort  required  to 
develop  effective  search  packages  for  patterns  of  these  types. 

The  concept  of  a single  tool  is  therefore  replaced  by  that  of  a set  of  packages  of 
techniques,  initially  small  but  continually  expanding  in  its  coverage,  and  useful  from  the 
beginning.  (Of  course,  certain  packages  may  accommodate  error  types  for  which 
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feature  extraction  directives  or  comparison  algorithms  are  similar.)  The  effect  of  this 
approach  is  that  an  enormous,  monolithic  manual  PE  process  has  been  broken  up  irto  a 
set  of  smaller  ano  much  more  manageable  processes,  each  concerned  with  one  or  a few 
error  types,  and  each  consisting  (conceptually)  of  two  subprocesses:  feature 

extraction  and  comparison.  An  example  of  the  application  of  such  a package  is  sketched 
in  the  next  section. 
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5.  /IN  EX  AMPLE 


The  application  of  a pattern-directed  search  technique  is  illustrated  by  a simple  and 
well-known  error  type  thal  can  be  characterized  as  "the  inconsistency  of  a data  value 
between  two  references."  The  error  is  first  represented  below  by  an  informal  but 
highly  abstract  pattern;  it  is  then  instantiated  into  two  familiar  examples,  and  finally  the 
second  case  is  used  to  illustrate  specific  search  techniques.  This  example  is  discussed 
in  more  detail  in  [Bis75]. 

The  pattern  is  the  following: 

1.  Operator  G reads  a value  from  or  writes  a value  into  cell  X. 

2.  Operator  F reads  the  value  from  cell  X. 

3.  Operator  G is  applied  before  operator  F 

4.  It  is  critical  (in  a protection  sense)  that  the  value  read  by  F is 
consistent  with  that  read  or  written  by  G. 

5.  Between  the  applications  of  G and  F,  the  value  in  X can  be  modified 
by  a process  other  than  that  in  which  F is  applied. 

This  pattern  can  be  instantiated  by  substituting  concrete  examples  for  the  abstract 
notions  of  operator"  and  "cell"  (see  Section  3).  One  common  subtype  of  this  error  is 
that  which  results  when  G and  F are  regarded  as  distinct  supervisor  procedures  and  X 
as  a global  variable.  For  example,  G and  F might  be  the  "checkpoint"  and  "restart" 
procedures  of  an  operating  system,  with  X being  the  file  used  to  store  a checkpointed 
computation.  If  a checkpoint  includes  sensitive  state  information  and  this  file  is 
modifiable  by  a user  between  the  checkpoint  and  restart  times,  then  that  user  could 
cause  his  computation  to  be  restarted  with  improper  privileges.* 


A 


• This  error  has  existed  and  has  been  exploited  in  the  third-generation 
operating  systems  of  more  than  one  manufacturer. 
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A second  common  subtype  results  when  G and  F are  low-level  operators  within  a 
single  user-callable  supervisor  procedure  and  X is  a parameter  of  that  procedure, 
passed  by  reference.  This  subtype  can  be  expressed  as  follows: 

1.  Operator  ij  (of  supervisor  procedure  P)  reads  or  writes  parameter  X 
(passed  by  reference). 

2.  Operator  F (of  P)  reads  parameter  X. 

3.  Operator  G is  applied  before  operator  F. 

4.  It  is  critical  that  the  value  read  by  F is  consistent  with  that  read  or 
written  by  G. 

5.  Between  the  applications  of  G and  F,  the  value  of  X can  be  modified 
by  a user  process. 

The  features  of  the  pattern  are: 

a)  Supervisor  procedure  (callable  by  user  procedures). 

b)  Parameter  passed  by  reference. 

c)  Operator  that  reads  or  writes  the  value  of  a given  parameter. 

To  search  for  instances  of  these  features,  "operator  that  reads  or  writes"  might  be 
instantiated,  for  example,  to  “code  that  fetches  or  stores."  The  context  of  item  C needed 
to  determine  relations  of  which  item  C is  a participant,  are  (1)  whether  it  reads  or 
writes  the  parameter,  (2)  its  location  in  the  (uninterpreted)  flow  of  control  of  the 
procedure,  and  (3)  if  it  does  read  the  parameter,  whether  or  not  the  consistency  of  its 
value  is  critical.  If  these  properties  are  Obtained  during  the  "feature  extraction”  step, 
then  the  subsequent  comparison”  step  need  only  determine  whether  the  relation 
"before"  holds  between  any  operator  reading  or  writing  any  parameter  and  another  that 
reads  the  same  parameter  and  is  "critical."  If  this  relation  holds,  a potential  error  is 
indicated. 

In  the  above  scheme,  most  of  the  work  is  done  during  feature  extraction,  while  the 
comparison  step  is  trivial.  Actually,  except  for  determining  the  criticality  of  an  operator 
that  reads  a given  parameter  (which  can  sometimes  require  considerable  analysis  by  a 
person  familiar  with  the  system),  feature  extraction  in  this  case  is  a well-defined 
procedure  and  can  be  entirely  automated  (if  reference  parameters  can  be  recognized). 

It  requires  a sophisticated  program,  of  course,  to  evaluate  flow  of  control.  If  final 
determination  of  flow  location  is  also  left  for  later,  then  feature  extraction  is 
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straightforward.  In  a program  of  moderate  size,  it  is  usually  easy  to  determine  by 
visual  inspection  whether  one  operator  can  occur  before  or  after  another.  This 
illustrates  the  tradeoffs  that  can  be  made  between  the  two  steps  of  the  PE  process  and 
the  flexibility  with  which  evaluation  techniques  can  be  designed  for  a given  error  type. 

As  an  initial  exercise  to  judge  the  feasibility  of  the  general  approach,  the  above 
pattern  was  applied  in  the  manner  just  described  to  portions  of  the  Mult ic s operating 
system  Since  Multics  is  written  in  a higher  level  language  (PL/1),  and  since  each  of  the 
pattern  features  has  a concrete  PL/1  representation,  there  were  no  difficulties  in 
identifying  and  extracting  instances  from  the  original  text.  A TECO  [Tec73]  program 
was  written  for  this  purpose.  Several  instances  of  errors  previously  unknown  were 
detected  and  verified. 
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6.  SUMMARY 


While  important  advance',  have  been  made  in  the  design  of  protection  mechanisms, 
they  are  not  generally  applicable  to  existing  general-purpose  operating  systems,  in 
which  there  is  a huge  investment  The  protection  aspects  of  such  systems  are 
notoriously  unreliable  and  the  security  risks  accompanying  their  use  in  certain 
environments  are  high  This  paper  has  addressed  itself  to  the  problem  of  "protection 
evaluation"  --searching  for  protection  errors  using  informal  static  methods,  i.e.,  methods 
that  depend  primarily  on  the  use  of  system  documentation  and  program  listings.  There 
is  a severe  shortage  of  anything  but  the  most  rudimentary  tools  for  this  task. 
Techniques  are  needed  that  can  be  applied  to  a wide  class  of  operating  systems  and 
that  do  not  depend  on  the  evaluator's  being  an  expert  in  the  field  of  security  and 
privacy. 

An  approach  has  been  proposed  in  which  formalized  patterns  are  used  to  direct  the 
protection  evaluation  task.  The  patterns  are  derived  from  the  analysis  of  errors 
previously  detected,  possibly  in  quite  different  systems.  The  report  discusses  the 
principal  components  of  a pattern-directed  methodOlogy--formulating  and  generalizing 
patterns,  instantiating  them  to  different  systems  and  functional  areas,  identifying 
instances  of  the  features  of  given  patterns  in  a targei  system,  and  comparing  the 
properties  of  the  instances  found  with  those  indicated  by  the  patterns.  It  concludes 
that  the  best  approach  is  to  develop  techniques  that  are  general-purpose  with  respect 
to  operating  systems  but  special-purpose  with  respect  to  error  types.  Among  the 
advantages  of  this  approach  are  that  the  techniques  are  simpler  and  can  be  ophmized  to 
particular  error  types,  the  approach  is  empirical  rather  than  theoretical,  its  payoff 
begins  sooner,  and  a set  of  such  tools  is  expandable  in  coverage  and  applicability. 

Examples  are  given  of  errors,  corresponding  patterns,  and  the  application  of  a 
pattern-directed  technique  to  the  search  for  errors  of  a particular  common  type. 
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